Bayesian parameter estimation through variational methods

We consider a logistic regression model with a Gaussian prior distribution over the parameters. We show that accurate variational techniques can be used to obtain a closed form posterior distribution over the parameters given the data thereby yielding a posterior predictive model. The results are readily extended to binary belief networks. For belief networks we also derive closed form posteriors in the presence of incomplete observations through the use of additional variational techniques. Finally, we show that the dual of the regression problem gives a latent variable density model, the variational formulation of which leads to exactly solvable EM updates.